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Issues  in  the  Background 

•  Data  to  Decision  vs.  Big  Data 

•  Mission  Scope 

•  Budget 

What  Does  This  Mean  for  Sensing  Research? 

•  We  Need  to  Rethink  How  We  Do  Sensors 
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What  Do  We  Need? 

1.  Intelligent  Hardware 

-  A  System  that  Facilitates  Decisions 

-  Adapts  Dynamically 

2.  Application  Specific  Hardware 

3.  Multi-Functional  Materials  for  Defence 

-  Not  multiple  applications  per  se 

-  Multiple  functions  per  material 

-  Enable  Intelligent  Hardware 
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Goals  for  Today: 

1.  Collaboration 

2.  Innovate 

3.  Motivate 
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TtCHtWOGV DRIVEN  Micro  and  Nano  Sensors  Team 


•  Alma  Wickenden  (ARL) 

•  Joe  Conroy  (ARL) 

•  Vishnu  Ganesan  (Case  Western) 

•  Kesshi  Jordan  (UC-Berkeley/San  Francisco) 

•  Alec  Koppel  (U  Pennsylvania) 

•  Yuan  Chen  (Princeton) 

•  Nick  Perkons  (Harvard) 

•  Daniel  Silversmith,  Kathryn  Schneider  (U  Maryland) 

•  Michael  Roberts  (U  Maryland  BC) 

•  Michael  Comparetto  () 
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•  Cyber  Physical  Sensing 
and  Control 

-  Robotics 

-  Human  Physiological 
Monitoring 

-  Piezo-MEMs 

-  CMOS  MEMs 

•  Collaboration 
Opportunities 


Overview 


Bio-Plausible  Control  Model 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 
UNCLASSIFIED 


^  US.  ARMY 

®  RDECOM 


TECHNOLOGY  DRIVEN  WARFIGHTER  FOCUSED 


Cyber  Physical 
Sensing  &  Control 


wing 

campanlforms 


Multi- 

sensory 

perception 


Program  Objectives: 

•  Develop  &  implement  bio- 
plausible  control  theory  for 
power  efficient,  reflexive 
mm-scale  robot  control  & 
sensor  array  information 
management 

•  Design,  fabricate  &  test 
bio-/neuro-inspired 
MEMS/NEMS  sensor 
modalities  with  minimized 
processor  requirements 

•  Determine  multisensory 
perception  models 
suitable  for  decision 
making  for  autonomous 
small  scale  system 
operation  & 

human/machine  interfaces 


Long  Term  Research  Goals: 


Implement  sensor  integration  logic  modeled  after  efficient  biological  systems 
revolutionary,  optimized  coordination  of  varied  physical  and  computational 1/ 
elements  in  fidelity-constrained,  extremely  low  power  sensor  systems.  / 


Flexible  Electronic 
or  Arrays 


Rob  Wood,  Harvard/MAST 


High  Density 
Multifunctional 
Neuroimaging 
Sensor  Arrays 


wing  contra! 
muscles 

* 


dascsnclng  - 
visual  information 


haltenc  cent 
■muscles 


raitere 

carrpaniforms 


Chan  et  al. 
Science,  1998 


FY09-10  DRI 


FY11  DRI 


Physical 

System 


ARL  Piezo  Wings  (fruitfly-scale) 


1;  I—  no 

5*|  (nonlinearity)  | 


“Slow- 

computing” 

algorithms, 

Richard 

Murray, 

Caltech/ 

ICB 


Power  efficient,  reflexive  multi-sensory  processing  will  increase  the  adaptability,  autonomy, 
functionality,  reliability,  and  efficiency  of  various  Army-relevant  power-  and  size-constrained  systems. 
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Power 

Systems 


Sensor 


Control 
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Systems 
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Autonomous  System  Technologies  provide  the  Soldier  with  superior 
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T^finologfesj 
breeding  a' new 
class  of  Sbfdi© 


Pt/PZT/Pt(6) 


TopAu 
contacts  (6) 


Si  stator 
wafer 


Bottom  Au  #  ’ 
contact 
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Autonomous  Systems  Enterprise 


Severely  Size,  Weight,  Power  and 
Processing  Constrained: 

•Comprehensive  Reexamination 
of  Systems 
•System  Integration 

•  Novel  Sensors 

•  Power  Conversion 
•Mobility  Constraints 
•Physics  at  the  Small  Scales 


Requires  a  Paradigm  Shift 
to  Integrated  Microelectronics 


Micro- 

l  fm 

Autonomous 
Jj  Sysjtem 
Te'chnolog 
breeding 
class  of  S<S 
set 


mm-Scale  Robotics 


■A.  *•  A 


ro-Fluidics  &  Optics  Packages 


ass 


Communications 


Si  stator  Pt/PZT/Pt(6)  TopAu 

wafer  /  contacts  (6) 


Bottom  Au  *  *  *  .  : '  • 

contact 


250  pm 


~  4 


Autonomous  System  Technologies  provide  the  Soldier  with  superior 


Caltech  Parallel,  Asynchronous  “Slow  Computing”  Logic 


3-axis 

Gyroscopes 


3-axis 

Accelerometers 


3-axis 

Magnetometers 


Temperature 

Sensors 
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Calibration 
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3D  Orientation 

Euler,  Matrix, 
Quaternion 
Sensors 
3D  Angular  Rate 
3D  Acceleration 
3D  Magnetic  Field 


Known  limitations: 
Large,  high  power,  limited 
sensor  input  formats,  data 
exchange  limits  (time, 
bandwidth) 


Standard  Serial  Control  Logi%UCLASSiFiEZECHNOLOGY  DRIVEN  warfighter  focused. 
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Big  Picture 


Sensors 


baiters 

Cftirpaniforms 


Control 


Actuation 
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Slow  Computing 


Traditional  -  requires  fast,  high  power 
processors 

Input  from  sensors 

i 


i 

Output  to  wing/motor 


Slow  computing  -  parallel  processing 
with  low  power,  slower  processors 
for  fast  control 


Input  from  sensors 


i 


Richard  Murray 


Output  to  wing/motor 
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Results 


More 

Noise 


Band  of  Attitude  Stabilization  in  Geodesic  Degrees 


Region  to  operate 
our  systems 


Less  information, 
faster  decisions 
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Percent  of  Simulations  Stable 
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Results:  Contrast  and  Noise 


High  Contrast  Environment  e  Low  Contrast  Environment 


Bioplausible  Feature  Tracker  Perfect  Bioplausible  Feature  Tracker  Perfect 


Controller  Type 
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Results:  Steady  State  Error 


Controller  Type 
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Results:  Contrast  and  Noise 


High  Contrast  Environment  e  Low  Contrast  Environment 


Bioplausible  Feature  Tracker  Perfect  Bioplausible  Feature  Tracker  Perfect 


Controller  Type 
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Structure  From  Motion 


Flight  Vehicle 


Feature  Extraction 
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Wearable  EEG  Development  and 
Testing  (WDT) 


The  current  neuroimaging 
modalities  do  not  allow 
assessment  of  brain  activities 
of  participants  performing  tasks 
involving  natural  movements 


ACHIEVEMENT  DESCRIPTION 


2 


Severely  limits  neuroergonomic 
studies  and  applications  in  real- 
world  environments 

, - if  ^ 

EEG  might  allow  freedom  of  ' 
movement  of  the  head  and  body, 
but  requires  advanced  on-line 
signal-processing  algorithms  to 
correct  movement  artifacts 

Barrier  Addressed: 

To  solve  the  lack  of  portable, 
user-acceptable,  and  robust 
systems  for  routinely  monitoring 
brain  and  body  dynamics  in 
complex  real-world 
environments  (B1 ,  B2) 


2010  MAIN  RESULT: 

Dry  foam-based  electrode 

Spring-loaded  probe  electrodes 

Wireless  data  acquisition  (DAQ)  circuit  board 

EEG  cap  (MINDO  16  helmet)  featuring  16  dry 

electrodes,  miniature  DAQ,  and  wireless 

telemetry 

Graphic  user  interface  on  mobile  devices  for 
data  logging 


wl\ 

-  S  * 

♦ 

continue  to  improve  the  DAQ  board 
Continue  to  improve  the  wearable  and  wireless 
dry-electrode  (WWD)  EEG  system 
Modify  the  MINDO  1 6  for  the  HD-Cog  TX1 7 
experiment  at  ARL 

Evaluate  the  quality  of  signals  acquired  by  the 
WWD  EEG  system 

Test  feasibility  of  using  the  WWD  EEG  system 
to  assess  cognitive-state  of  operators  in  MVS 
Develop  signal-processing  approaches  for  on¬ 
line  artifact  removal 


V. 


A  wireless,  wear-and-forget 
human  machine  interface  that 
can  allow  assessment  of 
brain  activities  of 
unconstrained  participants 
within  real  operational 
environments 

Transition: 

MoBI  for  detecting  changes 
in  vehicle  operator  alertness 
and  performance  (TE  1). 
Advanced  tools  for  MoBI  and 
neurocognitive  performance 
assessment  (TE  2  C4) 
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Can  we  augment  human 
performance  through  the 
WWD  EEG  monitoring  and 
processing? 


WWD  EEG  systems  are  easily  donned  and  doffed  for  cognitive  monitoring  in  real-world  environments 
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EEG  Filter  Design  for  Near-Real' 
Time  Signal  Processing 


•  CHALLENGE:  The  low  signal-to-noise  ratio  and  inconsistencies  due  to  inter-  and  intra¬ 
personal  variation  of  electroencephalography  (EEG)  data  prohibits  system  use  outside  of  a 
controlled  laboratory  environment. 

•  GOAL:  Develop  a  filter  that  allows  real-time,  frequency-specific  analysis  of  EEG  data  for  the 
practical  study  of  soldier  cognitive  function  on  a  single-trial  basis. 

•  APPROACH:  Investigate  filtering  methods  including  simple  and  multistage  band-pass  filters 
and  several  transforms  (Fourier,  Hilbert,  and  Wavelet)  using  MATLAB  simulation  and  testing 
to  determine  the  optimal  filter  design. 

•  INITIAL  RESULTS: 

•Adaptive  filter  produced  comparable  results  using  10  events  to 
current  neuroscience  method  using  32  events. 

•  Simulated  real-time  filtered  data  can  identify  some  obvious 
P300  responses  on  single-trial,  but  waveform  is  not  consistent 
enough  for  real-time  analysis  by  visual  inspection. 

•The  root  mean  square  deviation  values  are  insufficient  to 
evaluate  filter  performance. 

•  FUTURE  WORK: 

•  Further  processing  is  required  to  yield  generalizable 
results 

•  Validation  with  known  EEG  classifiers 
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The  result  of  band-pass  filtering  1  - 
100  hz  and  averaging  over  32  events 


By  narrow  band-passing  the  low  frequencies 
(circled),  the  SNR  is  greatly  improved. 


10  Trial  Adaptive  Filter  Output  Current  Neuroscience  Method  Output 
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Active  Research  Areas 


•  Miniaturization  of  EEG  electrodes  and  haptic  sensors 

•  Biocompatible  reversibly  adhesive  EEG  electrodes 
(“gektrodes”) 

•  Flexible  electronics  for  sensors 

•  Bioplausible,  multisensory  control/decision  making 
algorithms 

•  Development  of  robust  fMRI  sensors 

•  Self-organizing  sensor  arrays 


Enable  robust,  in¬ 
battlefield  detection 


Enable  enhanced 
sensing  and 
information  processing 
in  noisy  environments 


Flexible  electronics  research, 
John  Rogers  Group,  UIUC 
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Future  Collaborative  Research  Areas 


Novel  Sensors: 

-  Miniaturization  of  EEG  electrodes  and  haptic 
sensors 

-  Biocompatible  reversibly  adhesive  EEG  electrodes 
(“gektrodes”) 

-  Stretchable  electronics  for  sensors 

-  Development  of  robust  fMRI  sensors 

-  Self-organizing  sensor  arrays 


Enable  robust,  in¬ 
battlefield  detection 


•  Bioplausible,  multisensory  control/decision  making 
algorithms 

-  Autonomous  Systems 

-  Human  Physiological  Monitoring 

-  Computationally  efficient  algorithm  development  for 
robust,  noisy  applications 

•  SOAR 

-  CyberPhysical  Sensors  and  Control 

-  Adaptive  EEG 

TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 
UNCLASSIFIED 


Enable  enhanced 
sensing  and 
information  processing 
in  noisy  environments 
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Collaboration  Opportunities 


•  Research  of  Potential  Interest 

-  Cyber  Physical  Sensing  and  Control 

•  Robotics 

•  Physiological  Monitoring 

-  Piezo-MEMs 

-  CMOS  MEMs 

•  Facilities 

-  Clean-Room  Foundry 

-  Characterization  Facilities 

-  Robotics  Experimental  Facilities 

•  Mechanisms 

-  CQL 

-  SMART 

-  SOAR 

-  NSF 

-  Guest 
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8.  Wind  Science  and  Engineering 
Research  Center 

Texas  Tech  University 

Career  Atmospheric  scientist 

Learn  to:  Measure  hurricane  damage.  Students 

collect  data  about  hurricanes  and  tornadoes  by 

firing  debris  out  of  a  cannon. 

[PopSci  Sept.  2007] 

9.  WOODS  HOLE 
OCEANOGRAPHIC 
INSTITUTION 

Woods  Hole.  Massachusetts  [page  47] 

10.  Game  Design  Initiative 

Cornell  University 

Career  Videogame  designer 

Learn  to:  Create  your  own  game  at  the  first  Ivy  to 

offer  a  minor  in  game  design. 

[PopSci  Sept.  2008] 

11.  DIVISION  OF 
GEOLOGICAL  AND 
PLANETARY  SCIENCES 

California  Institute  of  Technology  [page  47] 

12.  Peruvian  Amazon  Field  Course 

New  College  of  Florida 
Career:  Rainforest  biologist 
Leam  to:  Follow  rare  animals  up  150-foot 
trees.  With  tree-climbing  pioneer  and  ecologist 
Meg  Lowman.  students  tag  animals  and  survey 
biodiversity.  [PopSci  Sept.  2010] 
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RESEARCH  LABORATORY 


Adelphi,  Maryland 


Massachusetts  Institute  of  Technology 
Career  Toy  designer 

Leam  to:  Build  toys.  Teams  of  six  get  a  theme 
and  $750  for  a  prototype. 

[PopSci  Sept.  2010] 

15.  REED  NUCLEAR 
REACTOR 

Reed  College  [page  50] 

16.  Lightning  Research  Lab 

University  of  Florida 

Careers:  Physicist,  engineer 

Leam  to:  Catch  millions  of  volts  from  the  sky 

by  using  rockets  to  draw  lightning  during 

passing  storms.  [PopSci  Sept.  2007] 


Science 

2011 

Most  Awesome 
Labs 


I  September  2011 

DESIGN  A  MICR0DR0NE 
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13.  U.S.  ARMY  RESEARCH  LABORATORY,  Aberdeen  Proving  Ground 
Careers:  Defense  researcher,  engineer 


Some  of  the  students  interning  on  the 
Micro-autonomous  Systems  Technology 
(MAST)  project  at  the  Army  Research  Lab 
in  Maryland  spend  their  summer  trying  to 
equip  soldiers  with  dozens  or  even  hundreds 
of  “insect'  robots  that  can  swarm  into  a 
bunker  or  cave  to  provide  a  remote  look 
inside.  “Working  in  silicon  at  the  fruit-fly 
scale,  they'll  cost  almost  nothing."  says 
Chris  Kroninger,  an  aeronautics  researcher 
specializing  in  MAST's  wings,  “and  they  can 
be  equipped  with  limited  sensor  capability 
that  can  be  a  first  warning  for  a  soldier.' 

Kroninger  says  the  robots  won't  be 
deployed  for  another  decade  at  least, 
and  there's  plenty  of  work  yet  to  be  done. 
That's  true  of  most  projects  given  to  the 
185  undergraduate  students  at  the  Army 
Research  Lab  every  year.  Whether  they 
analyze  body  armor,  develop  new  materials, 
or  create  miniaturized  sensors,  there's  more 
going  on  than  they  can  possibly  be  part  of. 

“A  lot  of  these  kids  come  in  with  a  deer- 
in-the-headlights  look,"  says  26-year-old 


researcher  John  Gerdes,  who  was  an  intern 
in  2006.  ‘But  the  ARL  does  such  a  good  job 
of  balancing  realistic  expectations  with  the 
proper  amount  of  mentoring  and  resources, 
it’s  almost  impossible  not  to  succeed.' 

Aaron  Harrington  came  to  the  ARL  as 
a  University  of  Maryland  sophomore  and 
wound  up  spending  two  more  summers 
there.  'I  didn't  know  that  research  was 
something  that  you  could  go  into,'  says 
Harrington,  who  is  now  24  years  old  and 
earning  a  master's  degree  in  aerospace 
engineering  from  his  alma  mater.  In  2008, 
Kroninger  assigned  him  to  photograph 
wings  flapping,  which  normal  cameras  will 
capture  only  as  a  blur.  Using  a  strobe  tight 
to  combat  the  blur  and  a  technique  called 
photogrammetry,  Harrington  used  two 
cameras  recording  video  at  30  frames  per 
second.  'If  you  had  two  cameras  and  knew 
how  far  apart  they  were  and  had  some  idea 
of  the  distance  from  the  test  bay,"  he  says, 
"you  could  use  triangulation  to  measure  the 
distance.  It  turned  out  to  be  very  accurate.’ 
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ROVING  RESEARCH. 

Students  Alec  Koppel,  Vishnu  Ganesan  and  Kesshi  Jordan 
demonstrate  the  miniature  robots  they  helped  develop. 
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TECHNOLOGY  DRIVEN  WARFIGHTER  FOCUSED 


Things  I  Wished  I  had 


We  are  Standing  on  the  Cusp  of  a  Substantial 

Paradigm  Shift  in  How  We  Collect,  Analyze,  and  Act 
Upon  Data 

•  How  do  we  collaborate  to  better  to  achieve  this? 

•  Are  there  any  innovative  approaches? 

•  How  do  we  motivate  a  concentration  in  research? 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 
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